Transformation Invariance in Pattern Recognition - Tangent Distance and Tangent Propagation

In pattern recognition, statistical modeling, or regression, the amount of data is the most critical factor a ecting the performance. If the amount of data and computational resources are near in nite, many algorithms will provably converge to the optimal solution. When this is not the case, one has to introduce regularizers and a-priori knowledge to supplement the available data in order to boost the performance. Invariance (or known dependence) with respect to transformation of the input is a frequent occurrence of such a-priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, \Tangent distance" and \Tangent propagation", which make use of these invariances to improve performance.